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ABSTRACT 



In 1990, the National Assessment of Educational Progress 
(NAEP) included a Trial State Assessment (TSA) ; for the first time in the 
NAEP's history, voluntary state-by-state assessments were made. The sample 
was designed to represent the 8th grade public school population in a state 
or territory. In 1996, 44 states, the District of Columbia, Guam, and the 
Department of Defense schools, took part in the NAEP state science assessment 
program. The NAEP 1996 state science assessment was at grade 8 only, although 
grades 4, 8, and 12 were assessed at the national level as usual. The 1996 
state science assessment covered three major fields: earth, physical, and 
life sciences. In Nevada, 133 students in 8 nonpublic schools. Results were 
not reported for public schools because of low participation level. This 
report describes the science proficiency of Nevada eighth^graders , compares 
their overall performance to students in the West region of the United States 
and the entire United States (using data from the NAEP national assessment) , 
presents the average proficiency for the three major fields, and summarizes 
the performance of subpopulations (gender, race/ethnicity, parents' 
educational level, Title I participation, and free/reduced lunch program 
eligibility) . To provide a context for the assessment data, participating 
students, their science teachers, and principals completed questionnaires 
which focused on: instructional content (curriculum coverage, amount of 
homework); delivery of science instruction (availability of resources, type); 
use of computers in science instruction; educational background of teachers; 
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and conditions facilitating science learning (e.g., hours of television 
watched, absenteeism) . On the NAEP fields of science scales that range from 0 
to 300, Nevada nonpublic school students had an average proficiency of 159 
compared to 162 for nonpublic school students throughout the United States. 

In Nevada, the average science scale score of students attending nonpublic 
schools (159) was not significantly different from that of nonpublic school 
students across the nation (162) . (DDR/NB) 
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I evada participated in the state-level National Assessment of Educational Progress 
(NAEP) for the first time in 1996. For the NAEP 1996 mathematics assessment, Nevada 
participated with both public and nonpublic school samples, and met the participation rate 
requirements’ for publication of mathematics results at grade 4 for both public and nonpublic 
school samples, and at grade 8 for their nonpublic school sample only. 

Nevada participated in the NAEP 1996 science assessment at grade 8 with both public 
and nonpublic school samples. The grade 8 public school sample did not meet the guidelines for 
publication, due to low participation rate (see Appendix A); however, the nonpublic school 
sample was sufficient to meet the guidelines for publication. 

The results of the NAEP 1996 science assessment in Nevada’s nonpublic schools are 
presented here. The total sample size for nonpublic schools is modest, and only those results 
based on pre-established NAEP minimum sample sizes are reported. In this report are tables 
showing the demographic composition of the sample, the participation rates for sample 
components, and the students’ average scale scores. 

A full set of Appendices is included: 

Appendix A Reporting NAEP 1996 Science Results 
Appendix B The NAEP 1996 Science Assessment 

Appendix C Technical Appendix 



1 To ensure comparability across jurisdictions, NCES has established guidelines for school and student participation rates. Appendix 
A highlights these guidelines, which are applied separately for public and nonpublic schools. For jurisdictions failing to meet the 
initial school participation rate of 70 percent for either public or nonpublic schools, appropriate results are not reported. 

Jurisdictions that exceed the 70 percent rate but fail to meet others of these guidelines are noted in tables and figures in NAEP 
reports containing state-by-state results. 
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OVERVIEW 



IVIonitoring the performance of students in subjects such as science is a key concern 
of the citizens, policy makers, and educators who direct educational reform efforts. The 1996 
National Assessment of Educational Progress (NAEP) in science assessed the current level of 
science performance as a mechanism for informing education reform. This science assessment is 
the first to be constructed on a new framework, and it is also the first to be given at the state 
level. In 1996, Nevada participated in the NAEP science assessment at grade 8 but only the 
nonpublic schools met the participation guidelines. This report contains those results. 

What Is NAEP? 

The National Assessment of Educational Progress (NAEP), the “Nation’s Report Card,” 
is the only ongoing nationally representative assessment of what America’s students know and 
can do in various academic subjects. 

NAEP is a congressionally mandated project of the National Center for Education 
Statistics, U.S. Department of Education. In 1990 Congress authorized a voluntary state-by-state 
NAEP assessment. The 1990 Trial State Assessment in mathematics at grade 8 was the first 
state-level NAEP assessment. Since then, state-level assessments have taken place in 1992 and 
1994 in reading (grade 4), in 1992 and 1996 in mathematics (grades 4 and 8), and in 1996 in 
science (grade 8). In 1996, 44 states, the District of Columbia, Guam, and the Department of 
Defense Education Activity Schools took part in the NAEP state assessment program; the state 
science assessment was at grade 8 only, although grades 4, 8, and 12 were assessed at the 
national level as usual. 

Results are provided only for group performance, since NAEP is forbidden by law to 
report results at an individual or school level. By making information on student performance 
available to policy makers, educators, and the general public, NAEP is an integral part of our 
nation’s evaluation of the conditions and progress of education. 
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What Was Assessed? 

The science assessment was crafted to measure the content and skills specified in the 
science framework for the 1996 assessment. 2 Two organizing concepts underlie the science 
framework. First, scientific knowledge should be structured so as to make factual information 
meaningful. The way in which knowledge is structured should be influenced by the context in 
which the knowledge is being presented. Second, science performance depends on knowledge of 
facts, the ability to integrate this knowledge into larger constructs, and the capacity to use the 
tools, procedures, and reasoning processes of science to develop an increased understanding of 
the natural world. Thus, the framework called for the NAEP 1996 science assessment to include 
the following: 

• Multiple-choice questions that assess students’ knowledge of important facts and 
concepts and that probe their analytical reasoning skills; 

• Constructed-response questions that explore students’ abilities to explain, integrate, 
apply, reason about, plan, design, evaluate, and communicate scientific information; 
and 

• Hands-on tasks that probe students’ abilities to use materials to make observations, 
perform investigations, evaluate experimental results, and apply problem-solving 
skills. 

The core of the science framework is organized along two dimensions. The first 
dimension divides science into three major fields: earth, physical, and life sciences. The second 
dimension defines characteristic elements of knowing and doing science: conceptual 
understanding, scientific investigation, and practical reasoning. Each question in the assessment 
is categorized as measuring one of the elements of knowing and doing within one of the fields of 
science (e.g., scientific investigation in the context of earth science). The framework also 
contains two overarching domains — the nature of science and the organizing themes of science. 
The nature of science encompasses the historical development of science and technology, the 
habits of mind that characterize science, and the methods of inquiry and problem solving. It also 
includes the nature of technology — specifically, design issues involving the application of 
science to real-world problems and associated trade-offs or compromises. The themes of science 
include the notions of systems and their application in the scientific disciplines, models and their 
functioning in the development of scientific understanding, and patterns of change as they are 
exemplified in natural phenomena. A fuller description of the framework is provided in 
Appendix B. 



2 Science Framework for the 1 996 National Assessment of Educational Progress. (Washington. DC: National Assessment 
Governing Board, 1993). 
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Who Was Assessed? 

Eighth-Grade Student Characteristics 

Table 1 provides demographic profiles of the eighth-grade nonpublic school students in 
Nevada, the West region, and the nation, based on data collected during the 1996 state and 
national science assessments. As described in Appendix A, the sample providing the state data 
and the sample providing the regional and national data are separate. 
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TABLE 1 — GRADE 8 



Profile of Students in Nevada, the West Region, and the Nation 





Demographic Subgroups 


Nonpublic Schools 


Percentage 







RACE/ETHNICITY 






Nevada 


White 


67 ( 8.0) 




Black 


5 ( 3.0) 




Hispanic 


17(4.0) 




Asian/Pacific Islander 


6 ( 3.7) 




American Indian 


5(1.1) 


West 


White 


67 ( 7.8) 




Black 


5(2.1) 




Hispanic 


19(6.1) 




Asian/Pacific Islander 


8(1.7) 




American Indian 


0 ( — ) 


Nation 


White 


80 ( 2.7) 


Black 


7(1.5) 




Hispanic 


9(2.1) 




Asian/Pacific Islander 


4 ( 0.8) 




American Indian 


1 ( 0.2) 


PARENTS’ EDUCATION 




2(1.3) 


Nevada 


Did not finish high school 




Graduated from high school 


13 ( 3.7) 




Some education after high school 


18 ( 3.6) 




Graduated from college 


62 ( 6.3) 




1 don’t know 


4 ( 1.8) 


West 


Did not finish high school 


2 ( 0.8) 
6 ( 2.2) 
12(2.4) 




Graduated from high school 




Some education after high school 




Graduated from college 


73 ( 5.5) 
7 ( 2.5) 




1 don’t know 


Nation 


Did not finish high school 


2 ( 0.3) 
10(1.1) 
17(1.8) 
66 ( 3.0) 
6(1.0) 




Graduated from high school 




Some education after high school 




Graduate from college 




1 don’t know 


GENDER 




47 ( 6.3) 


Nevada 


Male 

Female 


53 ( 6.3) 
51 (3.1) 


West 


Male 

Female 


49 ( 3.1) 
51 (1.8) 


Nation 


Male 

Female 


49 ( 1 .8) 



(continued on next page) 
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TABLE 1 — GRADE 8 (continued) 


Profile of Students in Nevada, the West Region, and the Nation 


State Assessment 





Demographic Subgroups 


Nonpublic Schools 


Percentage 







TITLE 1 
Nevada 


Participated 
Did not participate 


0 ( **'*) 
100 {****) 


West 


Participated 
Did not participate 


2 ( *—) 
98 ( **") 


Nation 


Participated 
. Did not participate 


7 ( 3.6) 
93 ( 3.6) 


FREE/REDUCED-PRICE LUNCH 
Nevada Eligible 

Not eligible 

Information not available 


0 ( **") 
0 ( *** *) 
100 (****) 


West 


Eligible 
Not eligible 

Information not available 


3 ( 1.8) 
34(13.7) 
64(14.3) 


Nation 


Eligible 
Not eligible 

Information not available 


7 ( 3.4) 
49 ( 7.7) 
44 ( 8.2) 



The standard errors of the statistics appear in parentheses. It can be said with about 95 percent confidence that, for each population 
of interest, the value for the entire population is within ± 2 standard errors of the estimate for the sample. In comparing two 
estimates, one must use the standard error of the difference (see Appendix A for details). The percentages for Race/Ethnicity may not 
add to 100 percent because some students categorized themselves as “Other.” **** Standard error estimates cannot be accurately 
determined. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment. 
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Schools and Students Assessed 

To ensure comparability across jurisdictions, NCES has established guidelines for school 
and student participation rates, highlighted in Appendix A. Jurisdictions failing to meet these 
guidelines are noted in tables and figures in NAEP reports containing state-by-state results. For 
jurisdictions failing to meet the initial school participation rate of 70 percent, results are not 
reported. 

For public schools in 1996, the weighted school participation rate before substitution was 
37 percent for Nevada; a level of 70 percent was required for publication of public school results. 
Nevada’s weighted school participation rate for nonpublic schools was 90 percent, meeting the 
guidelines for publication. Table 2 summarizes participation data for non public schools and 
students sampled in Nevada for the 1996 state assessment program in science. 3 

In Nevada, 8 nonpublic schools participated in the 1996 eighth-grade science assessment. 
The weighted school participation rate after substitution in 1996 was 90 percent for nonpublic 
schools, which means that the eighth-grade students in this sample were directly representative of 
90 percent of all the eighth-grade nonpublic school students in Nevada. 

In each school, a random sample of students was selected for the assessment. In 1996, 
on the basis of sample estimates, none of the eighth-grade nonpublic school population in 
Nevada was classified as having limited English proficiency (LEP), and 2 percent of the students 
had an Individualized Education Plan (IEP). Two percent of the students were excluded from the 
assessment on the basis of their IEP. 4 

In Nevada, 133 eighth-grade students from nonpublic schools were assessed in 1996. 

The weighted student participation rate was 91 percent for nonpublic schools. This means that 
the sample of eighth-grade students who took part in the assessment was directly representative 
of 91 percent of the eligible nonpublic school student population in participating schools in 
Nevada (that is, all students from the population represented by the participating schools, minus 
those students excluded from the assessment). The overall weighted response rate (school rate 
times student rate) was 82 percent for nonpublic schools. This means that the sample of students 
who participated in the assessment was directly representative of 82 percent of the eligible 
eighth-grade nonpublic school population in Nevada. 



3 For a detailed discussion of the NCES guidelines for sample participation, see Appendix A of this report or the Technical Report of 
the NAEP 1996 State Assessment Program in Science. (Washington, DC: National Center for Education Statistics, 1997). 

4 An IEP is a plan written for a student who has been determined to be eligible for special education. The IEP typically sets forth 
goals and objectives for the student and describes a program of activities and/or related services necessary to achieve the goals and 
objectives. 
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TABLE 2 — GRADE 8 



School and Student Participation in Nevada 



1996 



Nonpublic Schools 



SCHOOL PARTICIPATION 




Weighted school participation rate before substitution 


90% 


Weighted school participation rate after substitution 


90% 


Number of schools originally sampled 


10 


Number of schools not eligible 


1 


Number of schools in original sample participating 


8 


Number of substitute schools provided 


1 


Number of substitute schools participating 


0 


Total number of participating schools 


8 


STUDENT PARTICIPATION 




Weighted student participation rate after makeups 


91% 


Number of students selected to participate in the assessment 


151 


Number of students withdrawn from the assessment 


3 


Percentage of students who were of Limited English Proficiency 


0% 


Percentage of students excluded from the assessment due to 
Limited English Proficiency 


0% 


Percentage of students who had an Individualized Education Plan 


2% 


Percentage of students excluded from the assessment due to 


2% 


Individualized Education Plan status 
Number of students to be assessed 


146 


Number of students assessed 


133 


Overall weighted response rate 


82% 
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NAEP Science Results for Nevada 

T he NAEP 1996 state assessment program in science provides a wealth of information 
on the abilities and skills of the eighth-grade students in participating jurisdictions. The NAEP 
results presented in Table 3 are average scale scores on the NAEP science scale, representing 
what students in Nevada’s nonpublic schools know and can do in science. 

Reporting NAEP Science Results 

The NAEP Science Scale 

The NAEP 1996 science assessment spans the broad fields of science in each of the 
grades assessed. Because of the survey nature of the assessment and the breadth of the domain, 
each student participating cannot be expected to answer all the questions in the assessment since 
this would impose an unreasonable burden on students and their schools. Thus, each student was 
administered a portion of the assessment, and data were combined across students to report on 
the performance of eighth graders. 

Student responses to the assessment questions were analyzed to determine the percentage 
of students responding correctly to each multiple-choice question and the percentage of students 
achieving each of the score categories for constructed-response questions. Item response theory 
(IRT) methods were used to produce scales that summarized results for each of the three fields of 
science (i.e., earth, physical, and life) at each grade level. An overall composite scale also was 
developed separately at each of grades 4, 8, and 12 by weighting the separate scales based on the 
relative importance of each field of science in the NAEP science framework. Results presented 
in this report are based on this overall composite scale, which ranges from 0 to 300. 

The use of separate grade-specific reporting scales for the science assessment is 
consistent with the National Assessment Governing Board’s 1993 policy that future NAEP 
assessments be developed using within-grade frameworks and that scaling be carried out within 
grade. Because this science assessment was based on a new framework, and no comparisons 
with previous N AEP science assessments were possible, a new scale was developed. The ranges 
of the science scales (from 0 to 300) differ by design from the 0-to-500 reporting scales used in 
other NAEP subject areas; they were chosen to discourage inappropriate cross-grade 
comparisons and to minimize confusion with other common test scales. 
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The national average on the science scale is 150, including both public and nonpublic 
school students. The national average scale score reported here (162) is for students in nonpublic 
schools only. (Additional details of the scaling procedures are in Appendix C of this report and 
in the forthcoming NAEP 1996 Technical Report.) 

Science Achievement Levels 

A companion report, being issued by the National Assessment Governing Board, will 
present the NAEP 1996 science results in terms of achievement levels. As authorized by the 
NAEP legislation and adopted by the National Assessment Governing Board, the achievement 
levels are based on the Board’s judgments about what are reasonable performance expectations 
for students on the NAEP 1996 science assessment. The achievement levels for the NAEP 1996 
science assessment were adopted on an interim basis, indicating that they may be revised when 
other information becomes available, such as the fourth- and twelfth-grade results from the Third 
International Mathematics and Science Study (TIMSS). 

Interpreting NAEP Results 

Because the students whose average science scale scores reported here are sampled, and 
are not the entire population of eighth graders in a jurisdiction, the numbers reported are 
necessarily estimates. As such, they are subject to a measure of uncertainty, reflected in the 
standard error of the estimate. When the percentages or average scale scores of certain groups 
are compared, it is essential to take the standard error into account, rather than to rely solely on 
observed similarities or differences. Therefore, the comparisons discussed in this report are 
based on statistical tests that consider both the magnitude of the difference between the means or 
percentages and the standard errors of those statistics. 

The statistical tests determine whether the evidence, based on the data from the groups in 
the sample, is strong enough to conclude that the averages or percentages are really different for 
those groups in the population. If the evidence is strong (i.e., the difference is statistically 
significant), the report describes the group averages or percentages as being different (e.g., one 
group performed higher than or lower than another group). Rather than relying on the apparent 
magnitude of the difference between sample averages or percentages, the reader is cautioned to 
rely on the results of the statistical tests to determine whether those sample differences are likely 
to represent actual differences between the groups in the population. The statistical tests are 
discussed in greater detail in Appendix A. 

Mapping Items on the Scale 

This discussion and the item map illustration refer to eighth-grade students in the 
national assessment, whose scores may not resemble those of eighth-grade students in Nevada’s 
nonpublic schools. Students’ performance is summarized on the NAEP science scale which 
ranges from 0 to 300. Sample questions shown in Figure 1 illustrate the range of performance on 
the NAEP science scale for grade 8. Each sample question is one that is likely to be answered 
correctly by a student whose score is at or near the given percentile. 

16 



12 



THE NAEP 1996 STATE ASSESSMENT IN SCIENCE 



Nevada 



THE NATION'S 


FIGURE 1 


REPORT 

CARD 

199B 


rcaep 




Sample Questions Likely to Be Answered Correctly by Grade 8 
Students At or Near Selected Percentiles 


State Assessment 


Percentile 


Question 


10th 




Find typical yearly rainfall from a graph. (104) 


25th 




Explain the impact of fish death on an ecosystem. (1 27) 


50th 




Identify the effect of acid rain. (1 50) 


75th 




Understand where earthquakes occur. (172) 


90th 




Explain why lightning is seen before thunder is heard. (1 94) 



The value in parentheses represents the scale score attained by students who had a 65 percent probability of reaching a given level 
on a constmcted-response question (in italic type) or a 74 percent probability of correctly answering a 4-option multiple-choice 
question (in regular type). 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment. 

To illustrate the range of performance in more detail, questions from the assessment 
were “mapped” onto a 0 to 300 scale, as in Figure 2. The item map is a visual representation of 
the scale, showing selected questions in positions corresponding to their difficulty. The item 
map shows which questions a student of any particular ability is likely to answer correctly. The 
position of the question on the scale represents a dividing line: Students who attained scores 
greater than the score corresponding to the question’s difficulty are likely to answer it correctly, 
while students with scores below that degree of difficulty are less likely to answer it correctly. 

More specifically, students who scored below the scale score associated with a particular 
question had less than a 65 percent probability of earning a given amount of credit on a 
constructed-response question or less than a 74 percent probability of correctly answering a 
multiple-choice question. A small proportion of these students — those near but below the 
question’s position on the scale — may be more likely than not to answer the question correctly 
(between 50 and 65 or 74 percent). Such students are not considered “able” to answer the 
question, since they have not achieved sufficient consistency in their responses. 

Figure 2 is an item map for grade 8. 5 Multiple-choice questions are shown in regular 
type; constructed-response questions are in italic type. 6 An example of how to interpret the item 
map may be helpful. In this figure, a multiple-choice question involving interpreting a graph 
maps at the 136 point on the scale. This means that eighth-grade students with science scale 
scores at or above 136 are likely to answer this question correctly — that is, they have at least a 



5 Details on the procedures used to develop the item map are provided in the forthcoming NAEP 1996 Technical Report. The 
procedures are similar to those used in past NAEP assessments. 

6 The placement of constructed-response questions is based on (1) the “mapping” of a score of 3 on a 3-point scoring guide for short 
constructed-response questions and (2) the “mapping” of a score of at least 3 on a 4-point scoring guide and a score of at least 4 on a 
5-point scoring guide for extended constructed-response questions. 
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74 percent chance of doing so. 7 Put slightly differently, this question is answered correctly by at 
least 74 of every 100 students scoring at or above the 136 scale-score level. Note that this does 
not mean that students at or above the 136 scale score always answer the question correctly or 
that students below the 136 scale score always answer it incorrectly. 

As another example, consider the constructed-response question that maps at a scale 
score of 194. This question concerns the differing speeds of light and sound. Scoring of. 
responses to this question allows for partial credit by using a three-level scoring guide. Mapping 
a question at the 194 scale score indicates that at least 65 percent of the students performing at or 
above this point achieved a score of 3 (“Complete”) on the question. Among students with lower 
scores, less than 65 percent gave complete responses to the question. 



7 For constructed-response questions, a criterion of 65 percent was used. For multiple-choice questions, the criterion was 74 percent 
The use of a higher criterion for multiple-choice questions reflected the students’ ability to “guess” the correct answer from among 
the alternatives. 
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FIGURE 2 — GRADE 8 


Map of Selected Questions on the NAEP Science Scale 
for Grade 8 


State Assessment 



NAEP Scale 



Explain anise and prevention of crumbling of antient monument (21 3) 

Recognize part of cell that contains genetic material (205) ► 

Explain changes in appearance tmdnumber of hydra [W2) ► 
Understand forms of energy conversion (189) ► 


IPS 

— 192 

(90th paricantS*: 


b> 

◄ (206) Know which statement is consistent with theory of evolution 

◄ (194) Explain why lightning is seen before thunder is heard 


Understand trend of rainfall data on graph (184) ► 
Identify areas that hove worm summers and cold winters (1 80) ► 




◄ (184) Understand markings of contour map to find (firedion of river flow 

◄ (1 82) Understand which setup models the water cycle 


Understand where earthquakes occur (172) ► 
Know how pitdi is related to length (171) ► 
Measure pH of four types of so# (166) ► 
Devise experiment to investigate shadow <hanges[] 65) ► 


— — 174- 

(75th paramo*) 


M (172) Understand what happens when o magnet is placed inside a cod 

◄ (166) Understand movement of truck in relation to air 

◄ (1 63) Understand direction of movement after collision 


Identify effect of odd rain (150) ► 


(50lh panantih] 


◄ (1 58) Identify source of atmospheric oxygen 

< (153) Classify organism from characteristics 


Drnwinorbilsonmodelofsokr system [\39) ► 




< (148) Identify property of water that is most important for organisms 


Interpret graph showing seed production and rainfall (136) ► 


"i'iil-t.r.l/v ' 


< (135) Understand effect on density of adifing more salt to solution 


Explain advantages/rhsadvantages of planting near a stream (124) ► 
Interpret graph of revolution versus distance (121) ► 


• 125— — 

(25th pawiantfc) 


◄ (127) Explain impart of fkh death on ecosystem 

< (121) Identify best experimental setup 


Identify organs important for oxygen transfer (11 3) ► 




< (114) Identify property that results from processes of bring things 

< (113) Identify organisms that live in tropical rain forest 


Identify organism that produces its own food (89) ► 


||— '104 — . 
[10th pananda) 


◄ (104) Rnd typiad yearly rainfall from graph 

◄ (55) Determine whether markers are permanent or non-permanent 


Identify items that conduct electricity (28) ► 





> 



NOTE: Position of questions is approximate and an appropriate scale range is displayed for grade 8. 
Italic type indicates a constrocted-response question. Regular type denotes o multiple-choice question. 



Each grade 8 science question was mapped onto the NAEP 0-to-300 science scale. The position of the question on the scale 
represents the scale score attained by students who had a 65 percent probability of reaching a given score level on a constructed- 
response question or a 74 percent probability of correctly answering a 4-option multiple-choice question. Only selected questions are 
presented. Percentiles of scale score distribution are referenced on the map. 

SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment. 
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Results for Nevada 

The table below shows not only the average settle scores for students but also the 
distribution of their average scores at five selected percentiles. The distribution of the scores 
through these percentiles encourages the reader to consider the performance of the students in the 
various groupings (such as state, region, or nation) as overlapping ranges of heterogeneous 
performance, rather than as a simple monolithic average. As an example, consider the results 
below which show that, for Nevada’s nonpublic schools, students at the seventy-fifth percentile 
scored 178, while the average scale score for the West was 165. This means that at least twenty- 
five percent of Nevada’s nonpublic school students achieved scores higher than the average scale 
score for nonpublic school students in the West. 

• In Nevada, the average science scale score of students attending nonpublic schools 
(159) was not significantly different from that of nonpublic school students across 
the nation (162). 
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TABLE 3 — GRADE 8 


Distribution of Science Scale Scores for Students in Nonpublic 
Schools 


State Assessment 



Average 


10th 


25th 


50th 


75th 


90th 


Scale Score 


Percentile 


Percentile 


Percentile 


Percentile 


Percentile 



Nonpublic Schools 
1996 Nevada 


159(7.0)! 


122 (8.1)! 


138(6.7)! 


160(12.5)! 


178 (10.1)! 


197 (10.8)! 


West 


165(6.0)1 


128 ( 6.3)! 


148 ( 8.4)! 


167 ( 6.6)! 


184 ( 7.3)! 


198 ( 4.9)! 


Nation 


162(2.5) 


123(8.1) 


143(3.1) 


164 ( 3.0) 


182 ( 2.8) 


199 ( 2.1) 



The NAEP science scale ranges from 0 to 300. The standard errors of the statistics appear in parentheses. It can be said with about 
95 percent confidence that, for each population of interest, the value for the entire population is within ± 2 standard errors of the 
estimate for the sample. In comparing two estimates, one must use the standard error of the difference (see Appendix A for details). 
! Interpret with caution — the nature of the sample does not allow accurate determination of the variability of this statistic. 
SOURCE: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1996 Science 
Assessment. 



This report and the reports for other participating jurisdictions will be available on the 
NCES Web site (http://www.ed.gov/NCES/naep). A separate report describes additional eighth- 
grade science assessment results for the nation and the states, as well as the national results for 
grades 4 and 12. 8 

The Appendices that follow cover the NAEP 1996 science assessment for all 
jurisdictions. 



8 O’Sullivan, C.Y., C.M. Reese, and 3. Mazzeo. NAEP 1996 Science Report Card for the Nation and the States. (Washington, DC: 
National Center for Education Statistics, 1997). 
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APPENDIX A 



Reporting NAEP 1996 Science Results 

A.1 Participation Guidelines 

As was discussed in the Introduction, unless the overall participation rate for a 
jurisdiction is sufficiently high, the assessment results for that jurisdiction may be 
subject to appreciable nonresponse bias. Moreover, even if the overall participation rate 
is high, significant nonresponse bias may exist if the nonparticipation that does occur 
is heavily concentrated among certain types of schools or students. The following 
guidelines concerning school and student participation rates in the state assessment 
program were established to address four significant ways in which nonresponse bias 
could be introduced into the jurisdiction sample estimates. 

The first three guidelines describe the determination of whether a jurisdiction is 
eligible to have its results published. Guidelines 4- 1 1 describe conditions under which 
a jurisdiction’s published results will include a notation. Such a notation would indicate 
the possibility of bias in particular results, due to nonresponse from segments of the 
sample. Note that in order for a jurisdiction’s results to be published without notations, 
that jurisdiction must comply with all guidelines. (A thorough discussion of the NAEP 
participation guidelines can be found in the Technical Report of the NAEP 1996 State 
Assessment Program in Science .) 

Guidelines on the Publication of NAEP Results 



Guideline 1 — Publication of Public School Results 
A jurisdiction will have its public school results published in the NAEP 1996 
Science Report Card (or in other reports that include all state-level results) if 
and only if its weighted participation rate for the initial sample of public 
schools is greater than or equal to 70 percent. Similarly, a jurisdiction will 
receive a separate NAEP 1996 Science State Report if and only if its weighted 
participation rate for the initial sample of public schools is greater than or 
equal to 70 percent. 
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Guideline 2 — Publication of Nonpublic School Results 
A jurisdiction will have its nonpublic school results published in the NAEP 
1996 Science Report Card (or in other reports that include all state-level 
results) if and only if its weighted participation rate for the initial sample of 
nonpublic schools is greater than or equal to 70 percent AND meets minimum 
sample size requirements. 1 A jurisdiction eligible to receive a separate NAEP 
1996 Science State Report under guideline 1 will have its nonpublic school 
results included in that report if and only if that jurisdiction’s weighted 
participation rate for the initial sample of nonpublic schools is greater than 
or equal to 70 percent AND meets minimum sample size requirements. If a 
jurisdiction meets guideline 2 but fails to meet guideline 1, a separate NAEP 
1996 Science State Report will be produced containing only nonpublic school 
results. 



Guideline 3 — Publication of Combined Public and 
Nonpublic School Results 

A jurisdiction will have its combined results published in the NAEP 1996 
Science Report Card (or in other reports that include all state-level results) if 
and only if both guidelines 1 and 2 are satisfied. Similarly, a jurisdiction 
eligible to receive a separate NAEP 1996 Science State Report under 
guideline 1 will have its combined results included in that report if and only 
if guideline 2 is also met. 

Guidelines for Notations of NAEP Results 



Guideline 4 — Notation for Overall Public School 
Participation Rate 

A jurisdiction that meets guideline 1 will receive a notation if its weighted 
participation rate for the initial sample of public schools was below 85 percent 
AND the weighted public school participation rate after substitution was below 
90 percent. 



Guideline 5 — Notation for Overall Nonpublic School 
Participation Rate 

A jurisdiction that meets guideline 2 will receive a notation if its weighted 
participation rate for the initial sample of nonpublic schools was below 85 
percent AND the weighted nonpublic school participation rate after 
substitution was below 90 percent. 



22 
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Minimum participation size requirements for reporting nonpublic school data consist of two components: (1) a school 
sample size of six or more participating schools and (2) an assessed student sample size of at least 62. 
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Guideline 6 — Notation for Strata-Specific Public School 
Participation Rate 

A jurisdiction that is not already receiving a notation under guideline 4 will 
receive a notation if the sample of public schools included a class of schools 
with similar characteristics that had a weighted participation rate (after 
substitution) of below 80 percent, and from which the nonparticipating schools 
together accounted for more than five percent of the jurisdiction’s total 
weighted sample of public schools. The classes of schools from each of which 
a jurisdiction needed minimum school participation levels were determined 
by degree of urbanization, minority enrollment, and median household income 
of the area in which the school is located. 

Guideline 7 — Notation for Strata-Specific Nonpublic School 
Participation Rate 

A jurisdiction that is not already receiving a notation under guideline 5 will 
receive a notation if the sample of nonpublic schools included a class of 
schools with similar characteristics that had a weighted participation rate (after 
substitution) of below 80 percent, and from which the nonparticipating schools 
together accounted for more than five percent of the jurisdiction’s total 
weighted sample of nonpublic schools. The classes of schools from each of 
which a jurisdiction needed minimum school participation levels were 
determined by type of nonpublic school (Catholic versus non-Catholic) and 
location (metropolitan versus nonmetropolitan). 

Guideline 8 — Notation for Overall Student Participation 
Rate in Public Schools 

A jurisdiction that meets guideline 1 will receive a notation if the weighted 
student response rate within participating public schools was below 85 percent. 

Guideline 9 — Notation for Overall Student Participation 
Rate in Nonpublic Schools 

A jurisdiction that meets guideline 2 will receive a notation if the weighted 
student response rate within participating nonpublic schools was below 
85 percent. 
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Guideline 10 — Notation for Strata-Specific Student Participation 
Rates in Public Schools 

A jurisdiction that is not already receiving a notation under guideline 8 will 
receive a notation if the sampled students within participating public schools 
included a class of students with similar characteristics that had a weighted 
student response rate of below 80 percent, and from which the nonresponding 
students together accounted for more than five percent of the jurisdiction’s 
weighted assessable public school student sample. Student groups from which 
a jurisdiction needed minimum levels of participation were determined by the 
age of the student, whether or not the student was classified as a student with 
a disability (SD) or of limited English proficiency (LEP), and the type of 
assessment session (monitored or unmonitored), as well as school level of 
urbanization, minority enrollment, and median household income of the area 
in which the school is located. 

Guideline 11 — Notation for Strata-Specific Student Participation 
Rates in Nonpublic Schools 

A jurisdiction that is not already receiving a notation under guideline 9 will 
receive a notation if the sampled students within participating nonpublic 
schools included a class of students with similar characteristics that had a 
weighted student response rate of below 80 percent, and from which the 
nonresponding students together accounted for more than five percent of the 
jurisdiction’s weighted assessable nonpublic school student sample. Student 
groups from which a jurisdiction needed minimum levels of participation were 
determined by the age of the student, whether or not the student was classified 
as a student with a disability (SD) or of limited English proficiency (LEP), 
and the type of assessment session (monitored or unmonitored), as well as type 
and location of school. 
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A.2 NAEP Reporting Groups 

The NAEP state assessment program provides results for groups of students 
defined by shared characteristics — region of the country, gender, race/ethnicity, parental 
education, type of school, and participation in federally funded Title I programs and the 
free/reduced-price lunch component of the National School Lunch Program. Based on 
criteria described later in this appendix, results are reported for subpopulations only 
when sufficient numbers of students and adequate school representation are present. 

For public school students, there must be at least 62 students in a particular subgroup 
from at least 5 primary sampling units (PSUs). 2 For nonpublic school students, the 
minimum requirement is 62 students in a particular subgroup from at least 6 different 
schools. However, the data for all students, regardless of whether their subgroup was 
reported separately, were included in computing overall results for Nevada. Definitions 
of the subpopulations referred to in this report are presented on the following pages. 

Re gion 

Results are reported for four regions of the nation: Northeast, Southeast, Central, 
and West. The states included in each region are shown in Figure A.l. All 50 states 
and the District of Columbia are listed. Territories and the two Department of Defense 
Education Activity jurisdictions were not assigned to a region. 

Regional results are based on national assessment samples, not on aggregated state 
assessment program samples. Thus, the regional results are based on a different and 
separate sample from that used to report the state results. 
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Regions of the Country 


State Assessment 



NORTHEAST 


SOUTHEAST 


CENTRAL 


WEST 



Connecticut 


Alabama 


Illinois 


Alaska 


Delaware 


Arkansas 


Indiana 


Arizona 


District of Columbia 


Florida 


Iowa 


California 


Maine 


Georgia 


Kansas 


Colorado 


Maryland 


Kentucky 


Michigan 


Hawaii 


Massachusetts 


Louisiana 


Minnesota 


Idaho 


New Hampshire 


Mississippi 


Missouri 


Montana 


New Jersey 


North Carolina 


Nebraska 


Nevada 


New York 


South Carolina 


North Dakota 


New Mexico 


Pennsylvania 


Tennessee 


Ohio 


Oklahoma 


Rhode Island 


Virginia* 


South Dakota 


Oregon 


Vermont 

Virginia* 


West Virginia 


Wisconsin 


Texas 

Utah 

Washington 

Wyoming 



Note: The part of Virginia that is included in the Washington, DC, metropolitan area is included in the Northeast region; 
the remainder of the state is in the Southeast region. 



2 

For the State Assessment Program, a PSU is most often a single school; for the national assessment, a PS U is a selected 
geographic region (a county, group of counties, or metropolitan statistical area). 
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Gender 

Results are reported separately for males and females. 

Race/Ethnicitv 

The racial/ethnic results presented in this report attempt to provide a clear picture 
based on several sources. The race/ethnicity variable is an imputed definition of 
race/ethnicity derived from up to three sources of information. This variable is used for 
race/ethnicity subgroup comparisons. Two questions from the student demographics 
questionnaire were used in the determination of derived race/ethnicity: 



If you are Hispanic, what is your Hispanic background? 
0 I am not Hispanic. 

0 Mexican, Mexican American, or Chicano 
0 Puerto Rican 
0 Cuban 

0 Other Spanish or Hispanic background 



Students who responded to this question by filling in the second, third, fourth, or 
fifth oval were considered Hispanic. For students who filled in the first oval, did not 
respond to the question, or provided information that was illegible or could not be 
classified, responses to the question below were examined in an effort to determine 
race/ethnicity. 



Which best describes you? 

0 White (not Hispanic) 

0 Black (not Hispanic) 

0 Hispanic (“Hispanic” means someone who is from a Mexican, 
Mexican American, Chicano, Puerto Rican, Cuban, 
or other Spanish or Hispanic background.) 

0 Asian or Pacific Islander (“Asian or Pacific Islander” 
means someone who is from a Chinese, Japanese, Korean, 

Filipino, Vietnamese, or other Asian or Pacific Island background.) 

0 American Indian or Alaskan Native (“American Indian or 

Alaskan Native” means someone who is from one of the American 
Indian tribes, or one of the original people of Alaska.) 

0 Other (specify) 
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Students’ race/ethnicity was then assigned on the basis of their response. For 
students who filled in the sixth oval (“Other”) or provided illegible information or 
information that could not be classified, or did not respond at all, race/ethnicity was 
assigned as determined by school records . 3 

Derived race/ethnicity could not be determined for students who did not respond 
to either of the demographic questions and for whom a race/ethnicity designation was 
not provided by the school. 

The details of how race/ethnicity classifications are derived is presented so that 
the readers can determine the usefulness of the results for their particular uses. It should 
be noted that a nonnegligible number of students indicated a Hispanic background (e.g., 
Puerto Rican or Cuban) and indicated that a racial/ethnic category other than Hispanic 
best described them. These students were classified as Hispanic according to the rules 
described above. Also, information from the schools did not always correspond to 
students’ descriptions of themselves. 

Parents’ Highest Level of Education 

The variable representing level of parental education is derived from responses to 
two questions from the set of general background questions. Students were asked to 
indicate the extent of their mothers’ education: 



How far in school did your mother go? 

° She did not finish high school. 

° She graduated from high school. 

° She had some education after high school. 
° She graduated from college. 

° I don’t know. 



Students were asked a similar question about their fathers’ education level: 



How far in school did your father go? 

° He did not finish high school. 

° He graduated from high school. 

° He had some education after high school. 
° He graduated from college. 

° I don’t know. 



3 The procedure for assigning race/ethnicity was modified for Hawaii. See the Technical Report for the NAEP 1996 State 
Assessment Program in Science for details. 
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This information was combined into one parental education reporting variable 
through the following procedure. If a student indicated the extent of education for only 
one parent, that level was included in the data. If a student indicated the extent of 
education for both parents, the higher of the two levels was included in the data. For 
students who did not know the level of education for both parents or did not know the 
level for one parent and did not respond for the other, the parental education level was 
classified as “I don’t know.” If the student did not respond for either parent, the student 
was recorded as having provided no response. 

It should be noted that, nationally, approximately one-tenth of eighth graders 
reported not knowing the education level of either of their parents. 

Type of School 

Samples for the 1996 state assessment program were expanded to include students 
attending nonpublic schools (Catholic schools and other religious and private schools) 
in addition to students attending public schools. The expanded coverage was instituted 
for the first time in 1994. Samples for the 1990 and 1992 Trial State Assessment 
programs had been restricted to public school students only. For those jurisdictions 
meeting pre-established participation rate standards (see earlier section of this appendix), 
separate results are reported for public schools, for nonpublic schools, and for the 
combined public and nonpublic school samples. The combined sample for each 
jurisdiction also contains students attending Bureau of Indian Affairs (BIA) schools and 
Department of Defense Domestic Dependent Elementary and Secondary Schools 
(DDESS) in that jurisdiction. These two categories of schools are not included in either 
the public or nonpublic school samples. 

Note that eighth graders in the DDESS and Department of Defense Dependents 
Schools (DoDDS) 4 were assessed in 1996 as separate jurisdictions and reported as 
jurisdictions with public school samples only. 




4 



The Department of Defense Dependents Schools (DoDDS) refers to overseas schools (i.e., schools outside the United 
States). Department of Defense Domestic Dependent Elementary and Secondary Schools (DDESS) refers to domestic 
schools (i.e., schools in the United States). DoDDS and DDESS fourth grades were also assessed in science, for a 
special report. 
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Title I Participation 

On the basis of available school records, students were classified either as 
currently participating in a Title I program or receiving Title I services, or as not 
receiving such services. The classification only refers to the school year when the 
assessment was administered (i.e., the 1995 — 96 school year) and is not based on 
participation in previous years. If the school did not offer any Title I programs or 
services, all students in that school were classified as not participating. 

Free/Reduced-Price School Lunch Program Eligibility 

On the basis of available school records, students were classified either as 
currently eligible for the Department of Agriculture’s free/reduced-price lunch program 
or not. The classification refers only to the school year when the assessment was 
administered (i.e., the 1995 — 96 school year) and is not based on eligibility in previous 
years. If the school did not participate in the program or if school records were not 
available, all students in that school were classified as “Information not available.” 

A.3 Guidelines for Analysis and Reporting 

This report describes science performance for eighth graders and compares the 
results for various groups of students within this population — for example, those who 
have certain demographic characteristics or who responded to a specific background 
question in a particular way. The report examines the results for individual demographic 
groups and individual background questions. It does not include an analysis of the 
relationships among combinations of these subpopulations or background questions. 

Drawing Inferences from the Results 

Because the percentages of students in these subpopulations and their average 
scale scores are based on samples — rather than on the entire population of eighth 
graders in a jurisdiction — the numbers reported are necessarily estimates. As such, they 
are subject to a measure of uncertainty, reflected in the standard error of the estimate. 
When the percentages or average settle scores of certain groups are compared, it is 
essential to take the standard error into account, rather than to rely solely on observed 
similarities or differences. Therefore, the comparisons discussed in this report are based 
on statistical tests that consider both the magnitude of the difference between the 
averages or percentages and the standard errors of those statistics. 
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One of the goals of the science state assessment program is to estimate scale score 
distributions and percentages of students in the categories described in A.2 for the 
overall populations of eighth-grade students in each participating jurisdiction based on 
the particular samples of students assessed. The use of confidence intervals, based on 
the standard errors, provides a way to make inferences about the population average 
scale scores and percentages in a manner that reflects the uncertainty associated with the 
sample estimates. An estimated sample average scale score ± 2 standard errors 
approximates a 95 percent confidence interval for the corresponding population average 
or percentage. This means that one can conclude with approximately 95 percent 
confidence that the average scale score of the entire population of interest (e.g., all 
eighth-grade students in public schools in a jurisdiction) is within ± 2 standard errors 
of the sample average. 

As an example, suppose that the average science scale score of the students in a 
particular jurisdiction’s eighth-grade sample were 156 with a standard error of 1.2. A 
95 percent confidence interval for the population average would be as follows: 

Average ± 2 standard errors = 156 ± 2 x (1.2) = 156 ± 2.4 = 

156 - 2.4 and 156 + 2.4 = (153.6, 158.4) 

Thus, one can conclude with 95 percent confidence that the average scale score for the 
entire population of eighth-grade students in public schools in that jurisdiction is 
between 153.6 and 158.4. 

Similar confidence intervals can be constructed for percentages, if the percentages 
are not extremely large or extremely small. For extreme percentages, confidence 
intervals constructed in the above manner may not be appropriate, and accurate 
confidence intervals can be constructed only by using procedures that are quite 
complicated. 

Extreme percentages, defined by both the magnitude of the percentage and the size 
of the sample from which it was derived, should be interpreted with caution. (The 
forthcoming Technical Report of the NAEP 1996 State Assessment Program in Science 
contains a more complete discussion of extreme percentages.) 
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Analyzing Subgroup Differences in Averages and Percentages 

The statistical tests determine whether the evidence, based on the data from the 
groups in the sample, is strong enough to conclude that the averages or percentages are 
actually different for those groups in the population. If the evidence is strong (i.e., the 
difference is statistically significant), the report describes the group averages or 
percentages as being different (e.g., one group performed higher than or lower than 
another group), regardless of whether the sample averages or sample percentages appear 
to be about the same or not. If the evidence is not sufficiently strong (i.e., the difference 
is not statistically significant), the averages or percentages are described as being not 
significantly different — again, regardless of whether the sample averages or sample 
percentages appear to be about the same or widely discrepant. The reader is cautioned 
to rely on the results of the statistical tests rather than on the apparent magnitude of the 
difference between sample averages or percentages when determining whether those 
sample differences are likely to represent actual differences between the groups in the 
population. 

In addition to the overall results, this report presents outcomes separately for a 
variety of important subgroups. Many of these subgroups are defined by shared 
characteristics of students, such as their gender or race/ethnicity. Other subgroups are 
defined by the responses of the assessed students’ science teachers to questions in the 
science teacher questionnaire. 

In Chapter 1 of this report, differences between the jurisdiction and the nation were 
tested for overall science scale score and for each of the fields of science. In Chapter 
2, significance tests were conducted for the overall scale score for each of the 
subpopulations. In Chapters 3 through 6, comparisons were made across subgroups for 
responses to various background questions. 

As an example of comparisons across subgroups, consider the question: Do 
students who reported discussing studies at home almost every day exhibit higher average 
science scale scores than students who report never or hardly ever doing so? 

To answer the question posed above, begin by comparing the average science scale 
score for the two groups being analyzed. If the average for the group that reported 
discussing their studies at home almost every day is higher, it may be tempting to 
conclude that that group does have a higher science scale score than the group that 
reported never or hardly ever discussing their studies at home. However, even though 
the averages differ, there may be no real difference in performance between the two 
groups in the population because of the uncertainty associated with the estimated average 
scale scores of the groups in the sample. Remember that the intent is to make a 
statement about the entire population, not about the particular sample that was assessed. 
The data from the sample are used to make inferences about the population as a whole. 
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As discussed in the previous section, each estimated sample average scale score 
(or percentage) has a degree of uncertainty associated with it. It is therefore possible 
that if all students in the population (rather than a sample of students) had been assessed 
or if the assessment had been repeated with a different sample of students or a different, 
but equivalent, set of questions, the performances of various groups would have been 
different. Thus, to determine whether there is a real difference between the average 
scale score (or percentage of students with a certain attribute) for two groups in the 
population, an estimate of the degree of uncertainty associated with the difference 
between the scale score averages or percentages of those groups must be obtained for 
the sample. This estimate of the degree of uncertainty — called the standard error of 
the difference between the groups — is obtained by taking the square of each group’s 
standard error, summing these squared standard errors, and then taking the square root 
of this sum. 

In a manner similar to that in which the standard error for an individual group 
average or percentage is used, the standard error of the difference can be used to help 
determine whether differences between groups in the population are real. The difference 
between the mean scale score or percentage of the two groups — 2 standard errors of 
the difference — represents an approximate 95 percent confidence interval. If the 
resulting interval includes zero, there is insufficient evidence to claim a retd difference 
between groups in the population. If the interval does not contain zero, the difference 
between groups is statistically significant (different) at the 0.05 level. 

As another example, to determine whether the average science scale score of 
eighth-grade males is higher than that of eighth-grade females in a particular 
jurisdiction’s public schools, suppose that the sample estimates of the average scale 
scores and standard errors for males and females were as follows: 



Group 


Average Scale Score 


Standard Error 


Males 


148 


0.9 


Females 


146 


1.1 



The difference between the estimates of the average scale scores of males and females 
is two points (148 — 146). The standard error of this difference is 

V 0.9 2 + LF = 1.4 

Thus, an approximate 95 percent confidence interval for this difference is 
Mean difference ± 2 standard errors of the difference = 

2 ± 2 x (1.4) = 2 ± 2.8 = 2 - 2.8 and 2 + 2.8 = (-0.8, 4.8) 
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